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Talk Summary 



• Brief motivation on playtesting importance 

• Three workshops, not one 

• Workshops, not lectures 

NOTE: Resources at: 
www.playtestinqworkshops.com 
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Take-Away Points, Up Front 


• Super short iteration cycles 

• Hand's on practice, immediate feedback 

• Freedom to fail (not being graded) 

• Applicable to game design courses, 
studio/project courses, Game Creation 
Society meetings, .... 


GDC 




GAME DEVELOPERS CONFERENCE March 14-18, 2016 • Expo: March 16-18, 2016 #G0C16 


V 


ll[\ 



Credits: CMU Playtest Workshops 



Jessica Jodi Judeth MacKenzie Rachel 
Hammer Forlizzi Oden Choi Bates Moeller 
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Credits: Playtest Materials 


www.plavtestinqworkshops.com contents 
informed by academia and industry 


Test: Example 


Johann Sebastian Joust 



Photo by Brent Knepper in Pto Gute Fo bnk (http //gutefabnk comfoust htmO . 
J S Joust by Douglas Wilson for PS Move http wm iskhis! com 


Self-Assessment-Manikin Scales 

By: Hans Irtel 

An explanation of a pictorial rating system used to obtaii 
assessments of experienced emotions on the dimensions 
affective valence, dominance and arousal. 


0 Read More 



Why We Play Games: Four Keys to More Emotion 
Without Story 

By: Nicole Lazzaro 

XEODesign conducted an independent cross-genre research 
study on why people play games and identified over thirty 
emotions coming from gameplay rather than story. 


Motivations of Play in MMORPGs 

By: Nick Yee 

Yee built an empirical model of the underlying motivations of 
MMORPG players. This model provides a meaningful way to 
differentiate players from one another and a tool to explore in- 
game preferences and behaviors. 



;nt*gc of Respondents 


Like Emotion 
Games Theory 



GAME DEVELOPERS CONFERENCE March 14-18, 2018 - Expo: March 16-18, 2016 #GDC16 


GDC EDUCATION 


SUMMIT- 


V 


Usability and Playtesting 


Usability: efficiency, effectiveness, satisfaction 

Playtesting: "getting people to come play your 
game to see if it engenders the experience for 
which it was designed" - Jesse Schell, The Art of 
Game Design 

Definitions/context made into a 
packet which can be read-ahead 
material before workshops 
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Iterative Playtesting Process 


Your game 


Designing a play 
experience that 



gives you data 


Using that data 
to advance 
your design 
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Playtesting Skillset 


• Exploring a design 
space 

• Asking useful 
questions 

• Choosing a method 

• Executing effectively 
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Interpreting what 
you learn 

Iterating your game 

Making a case with 
data 



Handouts Important' 

Inside the box collect the ingredients that might go into creating your game: from game assets, to ideas, to personal experiences, 

to themes and styles. 


• Gives context to activities 

advice 


Playtesting with Plex Cards: Idea Generation with Stakeholders 

Try this within your design team to generate many ideas fast, or ask for participants outside of 
your team to do the exercise while you observe. Perhaps they will generate surprising new ideas. 
By having multiple people play with the same set of cards, you may find trends that your design 
team wants to explore or cliches that you want to avoid. 

Method One: Brainstorming 

1. Split into pairs. 

2. Each pair randomly draws a Plex Card and faces it heads up. This is the seed card. (You 
might pre-select the seed card, if you know that there is a particular type of playfulness 
that you want to explore.) 

3. Each player draws three more cards and keeps them in their hand. 

4. Player 1: start exploring an idea based on the seed card: What kind of games or 

PYnpripnrp*; romp to minrP On von havp a n^rtirnl^rlv momomhlp pyopopot^ with this 
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• Provides take-home 

Recon Activity: Composition Box Guide 

So, you've started brainstorming ideas for your game or experience! Use this visual organization tool, a Composition Box, to gather 
your design team's and stakeholders' ideas, inspiration, expectations and needs, and start developing your player experience goals. 
You might not be able to answer all of the questions below, and some will be more relevant to you than others— so choose where to 
focus your energy. To do this exercise you will need a poster, markers and printouts of images that inspire you. 


Start with what you know. Write a one- sentence description of your game or experience. 


^ Game Description 


Inspiration! 

Gather together games, art, stories, music, characters, and articles. Use images when possible. Try to articulate what inspires you 
about the examples. 

Consider: 

• Gameplay— mechanics, level design, feel, POV 

• Art— character design, art styles, consider art from different media and eras 

• Music— think game soundtrack, theme, feel, or era 

• Stories & Articles— What literature, ideas or current events inspire you? 


Experiences! 

Understanding the past and current experience of your stakeholders is essential to designing a game with meaning and 
resonance. Remember that your design team's experiences are also valuable fodder for design. 

Consider: 

• Play experience— How experienced are your stakeholders in your game-type? What kind of games do they have 
experience playing? 

• Technical experience— How technically experienced are your stakeholders? What technical expertise does your design 
team have? What skills does the design team want to learn or practice? 

• Personal experience— What personal experience do your stakeholders and design team have with the themes or subject 
matter of your game? What personal experiences might affect how they encounter your game and what type of 
expectations they might have for the game? 


Needs! 

What need(s) does your game satisfy? Is it a gap in the gaming market? Is it an educational need? An emotional or social need? 
Does it reach an underserved population? 

Consider: 

• What are your client's needs? 

• What are your players' needs? (What will/should keep them playing?) 

• What are the needs of other stakeholders? 

• Don't forget the needs of your design team. What do you want to get out of this experience? 


^ Player Experience Goals 


What do you want your player to feel or experience? By the end of this process, you should be able to articulate your player 

experience goals. 
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Playtest to Explore 


This workshop is for you if: 

• You are early in the development process. 

• You want to understand your players better. 

• You are struggling to find your game or 
experience. 
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Playtesting Methods in Explore 


• Observe if you know your target 
population. 

• Probe if you know an asset you'd like to 
use. 


• Test if you know your core mechanic. 

• Co-design if you know the impact you 
want to create. 


GDC 




GAME DEVELOPERS CONFERENCE March 14-18, 2016 • Expo: March 16-18, 2016 #GOC16 


V 


ll[\ 



Explore: Composition Box 


• Visualize many 
aspects of the 
experience 

• Captures 
perspective of 
varied team 
members 
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Explore: PLEX Cards 


Design for playfulness 

Credits: Andres Lucero & 

Juha Arrasvuori 

(funkydesignspaces.com/plex) 

Idea generation through 
mind maps triggered 
by card subset 
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PLEX-inspired Mind Maps 
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Playtest to Refine 



This workshop is for you if: 

• You are working on a playable prototype: 
lo-fi, hi-fi, subset of the experience, ... 

• You can articulate your player experience 
and/or design goals. 


GDC 
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Playtesting Methods in Refine 


• Observe to capture behavior 

• Think-aloud protocol to capture thoughts 
and decisions 

• Interview to capture motivations, feelings 
and attitudes 

• Surveys to capture structured data 
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"Good" Question Attributes 


• Important 

• Know what is core to your project 

• You only get to answer so many questions 

• Don't be tempted by what is easy 

• Answerable 

• Use familiar but unambiguous terms 

• Accurate 

• Be aware of bias: leading terms, social desirability 
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Refine: Practicing Observation 
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across 4 modified Uno! card games 

• Focus: social engagement (player-to-player) 
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Refine: Observation Lessons 


• Look for ACTION: vocalizations, gestures, etc. 

• Look for CHANGE: posture, focus (eye contact) 

• Situate within game, recording when action or 
change occurred 

• Withhold Interpretation 

• Do NOT Interrupt 

• Observation can lead to later directed surveys 
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Refine: Survey Bias 

(1) How fast was car A going 
when it turned right? 

(2) Did you see a stop sign? 


(x) How fast was car A going when it ran the stop sign? 

(y) Did you see a stop sign? 
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Refine: Survey Bias 

(1) How fast was car A going 
when it turned right? 

(2) Did you see a stop sign? 


Elizabeth F. Loftus, 
Leading Questions and 
the Eyewitness Report, 
Cognitive Psychology, 
1975 ( 7 ): 550-572 


35% answered yes to (2) 


(x) How fast was car A going when it ran the stop sign? 

(y) Did you see a stop sign? 

53% answered yes to (y) 
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Refine: Player Experience Goals 
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Refine: Peer Post-Its (+ and -) 
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Prove: Identifying Stakeholders 


Workshop opens with teams filling out rows across 
three columns: 

• Stakeholder 

• Claim 

• Evidence 
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Prove: Handout Excerpts 

Expert Panels: 

Ask experts to review your game. This may be a great way to get credible feedback. 


1. What is your claim? 


A/B Testing: 

Use A/B testing to test two possible versions of your game. Choose this method to help 
your design team make a difficult design decision. 


2. What is your timeline for receh 






Goal: 

Communicating what you've accon 


We will look at howto plan, execut 

* Expert panels 

* A/B testing 

* Pre/post testing 

* Time series tests, 

And how to articulate decisions ba 


1. What is your claim? 


2. Describe version j 


3. Describe version 
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4. Who a re your pla 


Pre/Post Tests: 

Testing before and after playing your game could be an effective way to playtest 
educational games or games that are intended to evoke a particular response. 


1. What is your claim? 


2. Who are your pi ay testers? 


4. How will you find your pla ytesters? 
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If You May Need Playtesting.. 


• Three workshops, to allow spread in a semester 


• Workshops, not lectures 

• Super short iteration cycles 

• Hand's on practice with immediate feedback 

• Freedom to fail (not being graded) 


• Applicable to game design courses, 
studio/project courses, Game Creation Society 
meetings, .... 
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More Playtesting Information 


• Explore, Refine, Prove Carnegie Mellon 
Workshops: http://plavtestinqworkshops.com 


• Professional interviews (some at above site) 

• Relevant books (some links at above site) 


Academic studies, e.g., ACM Digital Library (ACM 
CHI Play has relevant papers) 

Mike Christel, Entertainment 
Technology Center, christel@cmu.edu 



Carnegie Mellon University 

Entertainment Technology Center 
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